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Introduction 

The  purpose  of  this  funding  was  to  provide  support  for  two  meetings: 


•  The  26th  Annual  Conference  of  the  Cognitive  Science  Society.  This  meeting  is  the 
premier  venue  for  reporting  current  work  in  Cognitive  Science.  For  example,  it  is  the 
primary  meeting  where  new  results  on  analogical  reasoning  and  learning  are  reported. 
More  information  can  be  found  at  http://www.cogsci.northwestern.edu/cogsci2004. 

•  The  18th  International  Workshop  on  Qualitative  Reasoning.  This  meeting  is  the 
premier  venue  for  reporting  current  work  in  qualitative  reasoning.  For  example, 
results  on  human-like  spatial  reasoning,  reasoning  about  complex  devices,  and 
creating  intuitive  models  of  complex  non-linear  dynamic  systems  of  strong  interest  to 
DARPA  are  reported  there.  More  information  can  be  found  at 
http://www.qrg.northwestern.edu/qr04. 

Historically  DARPA  has  valued  and  supported  the  development  of  the  field  of  cognitive 
science  in  general  and  qualitative  reasoning  in  particular.  The  interdisciplinary  nature  of 
cognitive  science  has  provided  a  venue  for  researchers  from  computer  science, 
psychology,  neuroscience,  anthropology,  education,  and  philosophy  to  transcend  their 
disciplinary  boundaries  and  develop  the  leading  edge  theories  and  products  valued  by 
DARPA.  Examples  include  research  on  qualitative  reasoning,  analogical  reasoning,  and 
augmented  cognition.  Indeed,  the  focus  on  the  newly  reconstituted  IPTO  office  on 
Cognitive  Systems  indicates  the  depth  of  commitment  to  the  idea  that  we  must  create 
information  systems  that  are  more  like  people,  systems  that  know  what  they  are  doing, 
that  can  communicate  and  collaborate  with  human  partners,  and  can  reason  and  learn 
while  operating  for  months  and  years  at  a  time.  Cognitive  Science  provides  valuable 
clues  about  how  this  can  be  accomplished,  by  providing  scientific  results  and  insights 
based  on  humans  (and  sometimes  other  species). 

By  supporting  these  conferences,  we  gained  a  pipeline  into  a  community  of  scientists  and 
scholars  that  could  provide  valuable  insights  that  will  be  useful  for  future  DARPA 
programs  and  systems. 

The  funds  were  spent  as  budgeted,  and  the  financial  reports  have  already  been  submitted 
with  full  details.  The  proceedings  are  available  on-line  at  the  following  locations: 

Cognitive  Science  Meeting: 

http://www.cogsci.northwestern.edu/cogsci2004/ 

Qualitative  Reasoning  Workshop: 

http://www.qrg.northwestern.edu/qr04/ 

These  sites  have  papers  broken  down  into  individual  files,  since  that  maximizes  exposure 
to  search  engines  such  as  Google.  These  are  the  public  sites.  For  government  purposes, 
single  files  of  the  proceedings  may  be  found  at  http://www.cs.northwestem.edu/~forbus, 
specifically, 

http://www.cs.northwestern.edu/~forbus/Meetings  04/CogSci04  Proceedings  sans  front 

matter.pdf 
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http://www.cs.northwestern.edu/~forbus/Meetings  04/CogSci04  Front  mailer. pdf 

http://www.cs.northwestern.edu/QR04  Proceedings.pdf 

The  rest  of  this  report  provides  an  assessment  of  the  meetings. 

Assessment 

It  is  interesting  to  examine  some  of  the  work  presented  at  these  meetings  with  an  eye 
towards  potential  new  opportunities  for  DARPA  and  government  research  and 
development  more  generally.  In  this  section  I  will  describe  some  of  the  things  that  I 
found  especially  interesting  from  this  perspective.  These  opinions  are  my  own,  and  not 
those  of  the  Cognitive  Science  Society,  the  US  Government,  nor  should  they  be  taken  as 
representing  the  consensus  of  either  of  the  research  communities  involved. 


International  Workshop  on  Qualitative  Reasoning 

Qualitative  reasoning  research  has  often  focused  on  engineering  applications.  This  has 
led  to  significant  real-world  benefits,  e.g.,  the  use  of  model-based  reasoning  during 
design  (now  done  by  Ford  worldwide  for  automobile  electrical  systems,  with  other  car 
companies  rumored  to  be  moving  in  this  direction  as  well).  Some  excellent  work 
continues  in  this  vein  (cf.  the  papers  by  Bell  &  Snooke,  Struss,  and  Ramamoorthy  & 
Kuipers).  However,  there  is  now  a  healthy  broadening  of  what  is  being  explored,  leading 
to  some  intriguing  new  research  areas. 

•  Using  qualitative  modeling  for  education.  This  is  rapidly  becoming  a  new  major 
strand  in  the  community,  with  6  papers  out  of  3 1  involving  education  this  year. 
Qualitative  models  are  designed  to  be  close  to  human  mental  models  of  physical 
systems,  so  it  is  exciting  to  see  this  potential  start  to  become  realized. 

•  Qualitative  spatial  reasoning  remains  a  strong  thread  of  research,  with  four  papers 
ranging  from  basic  qualitative  shape  matching  to  use  of  qualitative  spatial 
techniques  to  analyze  numerical  data. 

•  The  use  of  qualitative  reasoning  for  modeling  ecosystems.  (3  out  of  3 1) 
Ecologists  coming  to  the  Workshop  are  skeptical  of  classical  modeling 
techniques,  arguing  that  they  require  more  quantitative  data  than  is  available, 
provide  a  false  sense  of  precision,  and  fail  to  highlight  patterns  of  behavior  that 
lead  to  insights.  This  is  consistent  with  the  arguments  made  earlier  by  researchers 
using  QR  for  modeling  nonlinear  dynamical  systems,  who  exploited  qualitative 
representations  of  phase  space  for  design  and  control,  e.g.,  the  work  of  Bradley, 
Zhao,  and  their  collaborators. 

•  Some  of  novel  uses  of  qualitative  modeling  are  being  explored,  including 
robotics,  virtual  reality,  natural  language  semantics,  and  using  compositional 
modeling  to  create  Bayes  nets  for  crime  scene  investigation. 

This  expansion  in  the  kinds  of  problems  tackled  is,  I  think,  a  very  healthy  sign. 
Qualitative  representations  may  turn  out  to  be  a  “sweet  spot”  in  the  space  of  formalisms 
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for  decision  problems  and  for  reasoning  about  policies,  as  well  as  for  modeling  human 
reasoning  about  physical  domains  and  engineering  artifacts. 


Annual  Meeting  of  the  Cognitive  Science  Society 

This  meeting  is  harder  to  summarize,  since  it  was  roughly  an  order  of  magnitude  larger  in 
terms  of  output  (265  papers  and  posters,  plus  150  member  abstracts,  versus  31  for  QR04), 
and  the  number  of  attendees  was  roughly  800.  Interestingly,  after  hovering  at  the  300 
mark  for  many  years,  over  the  last  three  years  we  have  seen  an  explosive  growth  in 
attendance.  The  center  of  gravity  of  the  Society  is  still  cognitive  psychology,  but  there  is 
still  an  intellectual  breadth  there  that  is  hard  to  find  in  any  other  conference  concerned 
with  cognition. 

We  chose  as  the  theme  this  year  higher-order  cognition.  This  theme  was  reflected  in  our 
choices  for  keynote  talks  and  invited  symposia,  but  to  our  surprise,  the  community 
responded  quite  strongly.  We  saw  far  more  papers  on  language  than  we  had  expected, 
for  example.  The  number  of  AI  papers  rose  somewhat  as  well,  through  recruiting 
members  of  the  AI  community  to  serve  on  the  Program  Committee.  Here  are  some 
highlights  of  the  conference: 

•  John  Anderson’s  Rumelhart  Award  talk.  Anderson’s  work  on  the  ACT-R  model 
now  stretches  from  hypotheses  about  specific  functions  of  neural  areas  to 
modeling  doing  distracting  tasks  while  driving  a  (simulated)  car.  The  range  and 
depth  of  his  research  programme  and  the  results  he  has  achieved  are  impressive. 
This  award  was  well-deserved. 

•  The  impact  of  culture  on  cognition  is  becoming  a  hot  topic.  As  Doug  Medin’s 
keynote  talk  observed,  most  researchers  use  college  students  as  their  main  source 
of  participants  in  experiments,  and  assume  that  results  generalize.  There  is  now  a 
growing  body  of  evidence  that  they  do  not,  and  that  conceptual  structures  and 
reasoning  can  vary  quite  significantly  across  cultures.  A  symposium  on  Language 
and  Thought  explored  issues  such  as  how  causal  verbs  varied  across  languages, 
and  what  this  might  say  about  how  causal  events  are  represented  non- 
linguistically. 

•  Gesture  has  become  an  important  window  for  looking  into  cognition.  Susan 
Goldin-Meadow’s  keynote  talk  described  her  research  indicating  that  mismatches 
between  what  is  said  in  speech  and  what  is  communicated  via  gestures  can 
mislead  students,  when  done  by  a  teacher,  and  can  be  a  sign  of  incipient 
conceptual  change,  when  done  by  a  student.  (This  is  an  important  result  to  keep 
in  mind  when  designing  embodied  conversational  agents  for  interfaces.)  A 
symposium  organized  by  Greg  Trafton  of  NRL  brought  together  researchers  from 
different  disciplines  to  explore  how  gestures  are  used  in  spatial  thinking. 

•  A  symposium  of  qualitative  reasoning  researchers  provided  a  bridge  between  the 
QR  community  and  the  Cognitive  Science  community.  Creating  such  bridges  via 
co-location  was  the  reason  why  we  took  on  organizing  both  meetings  at  the  same 
time.  There  was  considerable  interest  by  cognitive  science  attendees  in  using  QR 
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techniques  as  formal  representations  for  human  mental  models.  We  hope  that  this 
will  lead  to  more  joint  research. 

•  The  cognitive  science  community  was  made  more  aware  of  large-scale 

representational  resources  that  have  been  developed  for  research  purposes  by  a 
symposium  that  included  George  Miller  (WordNet),  Charles  Fillmore 
(FrameNet),  Doug  Lenat  (Cyc),  and  Martha  Palmer  (VerbNet).  Lenat’s 
description  of  ResearchCyc,  and  its  near-tenn  availability  to  the  research 
community,  caused  considerable  excitement.  These  resources,  in  the  hands  of  the 
AI  and  Cognitive  Science  communities,  could  lead  to  an  inflection  point  in  the 
kind  of  research  that  can  be  carried  out.  It  is  impressive  that  these  researchers 
have  carried  on,  for  over  a  decade  in  some  cases,  despite  the  current  focus  on 
short-range  results  in  funding. 

Summary 

Both  meetings  provided  valuable  scientific  results,  some  of  which  directly  bear  on 
DARPA-related  activities.  We  hope  that  the  assessment  above  will  excite  the  reader 
enough  to  delve  into  the  proceedings,  for  there  are  many  gems  in  them  that  are  worth 
your  time. 
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